This is a brief introduction to the Open Transport Virtual Server and Open Transport Virtual Client programs found in this folder.
The Virtual Server is an attempt to provide a piece of sample code which uses the native Open Transport API and is complex enough to demonstrate real issues an application developer will need to deal with. It also demonstrates the speed of Open Transport when used properly.
This version of the Virtual Server simply opens a listening endpoint and as many accepting endpoints as you want using TCP. It waits for an inbound connection request, accepts the connection, and hands it off to an accepting endpoint. The accepting endpoint waits for a 128 byte "request" packet, then returns a predefined amount of data from memory to the client. It then does an orderly release and puts the endpoint back into its idle queue.
It's fast. Running on Open Transport 1.1.1, a Power Mac 7100/80 on a 10 Mbit ethernet using 8K downloads can almost saturate the wire. Using a 200 MHz Power Mac on the same LAN with 1K downloads the server sustains better than 300 connections per second.
In the development of this, several problems were found in Open Transport which we hope to fix soon after the Open Transport 1.1.2 release. The program checks the Open Transport version, and if it finds something beyond 1.1.2, it automatically stops using workarounds which slow it down by about 18%. On an Open Transport prototype with these fixes, the same 200 MHz Power Mac sustained more than 400 connections per second.
The Rules for Speed:
Here are the rules if you want to go fast using Open Transport 1.1.1 on MacOS 7.5.
1) Don't use threads (until System 8).
2) Use the native Open Transport API and endpoints in asynchronous, blocking mode, with notifiers installed.
3) Do everything possible directly from the notifier. Do not set flags in the notifier telling the event loop to do something.
4) Never turn off interrupts. Use atomic operations and atomic LIFO queues for synchronization wherever necessary.
5) Use the new "tilisten" module for listening endpoints.
6) Use no-copy receives and Ack Sends whenever possible.
7) Reuse endpoints rather than closing them and opening new ones.
In the development of this sample, a problem was identified in TCP which forces the program to pass through the event loop once during the lifetime of each connection before an orderly release is done. This problem will be fixed in the next release of Open Transport after 1.1.2. The cost of this single pass through the event loop, in terms of connections per second, is about 18%. This should make it clear just how important it is to do everything possible within the notifier.
Sample Program Highlights:
This sample, unlike earlier ones, is a real, complete Macintosh application, as opposed to an SIOW application or MPW tool. Looking at the C files, you will find that the networking code is located at the top of the file, and well commented. The lower portion of the file is the Macintosh program wrapper which contains pieces not specifically related to networking. This sample attempts to demonstrate best networking practices. There are many better samples for general Macintosh application programming.
Looking at the samples, please notice the following:
1) The sample code has a debug and production mode. The idea here is for the program is for the program to recover, whenever possible, in production mode, but to exit with an alert in debug mode whenever anything new and unexpected occurs.
2) The vast majority of the sample code is invoked directly from the notifier. The sample code never turns interrupts off. OTAtomicAdd32, OTAtomicSetBit, OTAtomicClearBit, OTLIFOEnqueue, and OTLIFODequeue are used to avoid the need to turn interrupts off. All of these operations are atomic and therefore interrupt safe. Also notice the usage of OTEnterNotifier and OTLeaveNotifier which is used to synchronization processing of deferred T_LISTEN events with the listener endpoint's notifier.
3) Originally, the program ran so fast that the system event loop never got time. That made it impossible to quit at times. To avoid this, a 3 second heartbeat timer was added. If the event loop doesn't run at least once ever three seconds, the server "throttles back" just enough to get the event loop to fire, allowing the user the ability to stop or adjust the program.
4) Most non-networking work, such as updating the statistics window, is done in via the event loop. This way it has little impact on performance which is driven by the notifier.
5) Creating this program, a handful of problems were identified in OT 1.1.1. They are all documented in comments within the source, and workarounds are provided for all of them. After OT 1.1.2, the same sample will run faster. However, as it is, it will run reliably and quickly on OT 1.1.1 and 1.1.2.
6) The tilisten module, provided originally in OT 1.1.1 (both in the system and as a sample module) is used here to serialize inbound connection requests. This off loads more of the connection queue management down to the Open Transport kernel, where it can be dealt with most efficiently. This avoids the complexity of the "8 Step listen/accept/disconnect" method which has been documented at length elsewhere. It also allows better continuous forward progress when a server is heavily loaded with inbound traffic.
7) The server demonstrates a reliable recovery mechanism via the "broken endpoint LIFO". While the problems which caused its creation have all been worked around, it is left in the sample as a demonstration of how to handle unexpected problems. The ultimate recovery mechanism is to close the endpoint and open a new one to replace it. This is was that LIFO is used for.
8) The server uses Ack Sends to avoid having data copied on outbound sends. This comes with a cost, though, which is that an endpoint cannot be closed until all such outstanding sends have received a T_MEMORYRELEASED event. Also, the program cannot quit until this is true for all endpoints. In order to do this, some logic was added to the endpoint closing code to protect against such problems. Using Ack Sends is a good idea, but developers must be cautious about such problems and code to prevent them.
9) The program also uses no-copy receives, although they do not present any of the challenges which Ack Sends provide.
10) Much like Ack Sends, the program cannot go away while there is an outstanding OTAsyncOpenEndpoint in progress. There is also code to protect against this.
11) The program uses Open Transport "helper functions" wherever possible. No sense in recoding anything which comes for free.
12) The processing of kOTLookErrors is something which is not initially obvious to many new XTI programmers. This sample shows many places where they should be expected.
What's Next:
This sample is the first of several which we hope to provide to Open Transport developers. It will be improved as we get feedback, and as the author has time to finish cleaning up the Macintosh application wrapper. It will be followed up by versions which access disk storage to retrieve requested information, demonstrating more complex synchronization problems, and which use ADSP as well as TCP, to demonstrate transport independence.
